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Abstract 



This paper presents the Minneapolis Public School’s (MPS) method of measuring school and 
student performance. MPS employs a multi-faceted system that both captures and accounts for the 
complexity inherent in a large urban school district. This system incorporates: (a) a hybrid model of 
critical indicators that report on level of performance, cross-cohort changes, growth or gain scores, and 
value-added measures; (b) specific standards, set in a collaborative manner by key stakeholders, and 
standards were then aligned with MPS criteria and real-world consequences; and (c) a cycle of continuous 
School Improvement Planning. 

The MPS hybrid model includes a set of pre-agreed upon, pre-selected indicators that were 
established in concert with stakeholders who are internal and external to the Minneapolis Public Schools. 
Moreover, this paper provides a practical application of how value-added indicators can be and should be 
included in any fair accountability model designed to evaluate the effectiveness of schools. The expanded 
growth model, referred to as value-added, addresses the differential balance of student characteristics 
across MPS schools such as: poverty, race, family composition, special education status, Limited English 
status, and neighborhood poverty concentration. The value-added approach described here is the backbone 
of setting up a flexible, yet equitable measurement system that can account for important student 
characteristics (associated with student achievement) and assist in determining the extent to which teachers 
and schools “add value” to student performance. 

Evaluating the overall performance of schools based upon a set of indicators allows the 
Minneapolis Public School’s system to evaluate the district’s effectiveness in meeting the needs of all its 
students. Moreover, the overall framework allows the district to place schools along a continuum that relies 
on both rewards and sanctions. In fact, by using the described framework schools can "earn financial 
rewards” called Quality Performance Awards to maintain and expand upon their excellent performance. 
Additionally, in extreme cases schools can be identified that "need prescriptive, corrective action,” which 
may include the option of school reconstitution through the district’s “fresh start” provisions of the teacher 
and principal contracts. 

Finally, a complete School Improvement Planning cycle is discussed. Continuous school 
improvement is based on detailed information reports provided to all school sites titled, “School 
Information Reports.” These documents set the stage for important site-based analysis and planning to 
occur. Samples of these yearly documents are available via the world-wide-web at: 
http://www.mpls.k 1 2.mn.us/REA (click on School Information Reports). Hopefully, this paper will be 
useful to practitioners, administrators or researchers interested in identifying and combining school 
performance indicators to make high stakes accountability decisions for school districts. 



AERA_Performance continuum-2000 



1 



04/18/00 



Measuring school performance to improve student achievement 

In order for a school incentive program to be effective, that is, to have motivational impact 
on schools which results in improved student learning, it must be perceived as fair and 
worthwhile, and it's criteria must be understandable and readily communicated. (John 
May, in Manedeville, 1994). 

Evaluating School Performance. 

No matter where you turn, you cannot escape the pervasive discussions, debates, and hyperbole 
that surround the topic of evaluating student outcomes. Further, attempting to understand the role schools 
and districts play in accelerating or decelerating student performance can be daunting. Although 
evaluating school and district performance is a given in today’s educational environment, there are still 
varying perspectives on how to best meet the challenge (Millman & Schalock, 1997). In fact, an elaborate 
array of state-wide and district-wide models have been proposed. These evaluation models are often put in 
place under the guise of addressing such things as: (a) Accountability; (b) Communicating performance to 
key stakeholder including the community at large; (c) Effecting change for poor performing sites or 
creating the impetus for self-study and self-improvement among all schools; and increasingly to (d) 
Identify innovative and powerful programs so that they may be replicated and shared with sites that are 
performing well below expectations, as well as to reward high-performing schools. 

It should come as no surprise that a multitude of systems have been developed to evaluate school 
effectiveness. Many state and local models rely on reporting level indicators of student test performance 
(e.g., SAT or ACT scores, or some other mean score). However, as it will be discussed later, these are 
imperfect methods for determining the inherent value or effectiveness of instructional programming within 
schools. Recent developments have pushed the frontier toward more sophisticated statistical analyses to 
help get the answer “right” when trying to attribute school effects v. non-school effects on student 
performance (Meyer, 1996). The need for more sophisticated analyses has arisen out of the need to 
account for the complex set of interactions that occur between students who enroll in programs and school 
and district ability to meet the needs of those students. To this end, more systems are relying on 
educational outcome indicators to assess the efficacy of educational programming (Meyer, 1996). Thus, 
student outcomes or student performance on clearly identified indicators have become the primary method 
for evaluating and holding schools and school systems accountable. 

Criteria for performance indicators and accountability systems . 

Meyer (1996) identified three critical criteria for evaluating the usefulness of performance 
indicators. First, school performance indicators that are included in any accountability system must assess 
the types of skills demanded by society. In the state of Minnesota there are some clear benchmarks all 
students must meet. In 1996 the state of Minnesota implemented a series of graduation requirements that 
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have been implemented incrementally within the state. For example, beginning with the graduating class 
of 2000, students must pass Basic Skills Tests (MBST) in the areas of reading and math. For the class of 
2001 and beyond, students must pass MBSTs in the areas of reading, math and writing. Students must 
reach a passing score in order to be eligible to receive a high school diploma. In addition, the state has put 
in place a series of high standard tests referred to as the Minnesota Comprehensive Assessment (MCA) at 
grades 3, 5, 10, and IP. These tests cover high-end content and knowledge in the areas of reading, math, 
and writing. This two-tiered system provides clearly defined outcome measures that have allowed the MPS 
to align curricula and district assessments to plot student performance and trajectories toward basic and 
high standard criteria. 

The second characteristic of a quality school performance indicator is its ability to accurately 
measure performance with respect to the outcome it purports to measure (Meyer, 1996). That is, the 
indicators that are selected must meet a standard in which they cannot be “corrupted.” Thus, the 
instruments must be valid and reliable for the purpose for which they are selected. The administration of 
such instruments must occur following rigorous standardized procedures. Meyer (1996) describes 
additional concerns about the ability to corrupt an accountability system. For example, when level 
performance indicators are used alone, there may be a disincentive for districts to include students at the 
lower margins (e.g., special education and ESL students) to avoid “lowering” the overall mean test scores. 
Obviously this behavior is unethical and creates a system in which some students become “invisible” 
within the district, resulting in no accountability for their performance. In effect, districts could become 
disingenuous in setting policies and begin to test only “higher performing” students so as to corrupt student 
performance reports. Additional policies, such as retention of students at specific grade levels can further 
distort level performance, as students remain at the same grade-level for a second year and are compared to 
norm groups based on grade. Collateral damage also can occur as teachers then begin to bid out of “low- 
leve!” and into “high level” schools. Finally, very few if any of these level models account for the high 
correlation of post-test performance based on pre-test performance. Indeed, few systems collect student 
achievement on a yearly basis to compare actual improvement across time. 

Finally, over and above the “corruptible” concern, performance indicators selected must measure 
the unique contribution schools “add” for each measured outcome. Although many systems rely on 
average test score performance to rank order or evaluate school performance, mean scores are highly 
flawed indices of school effectiveness (Meyer, 1994). Despite the flawed nature of equating school 
performance with point-in-time mean performance, many districts, newspapers, real estate brokers, and 
families resort to evaluating or making judgements about school effectiveness based on level data. 



1 The state of Minnesota currently has high standard MCA tests at 3, 5, and 10 and will be adding two additional 
MCA tests in reading (grade 10) and math (grade 1 1) during the 2000-2001 school year. 



For example, take two schools — School A and School B — that currently report overall student 
performance at the 80^ percentile (a high performing school). Despite its seductive, simplistic approach, 
level indicators do not provide adequate information on how a student, school, or district reached such a 
level of performance. In this example there is no way to account for how these students or schools were 
performing the previous year, nor is there any accounting of whether student performance has stagnated, 
accelerated or decelerated to reach this reported level. These are critical aspects that must be addressed in 
any accountability system. This incomplete view of performance can result in poor decision making at best 
and all but ensures that school districts will draw false conclusions when trying to determine the 
effectiveness of school policies on student outcomes (Meyer, 1996). 

In addition to the critical criteria for performance indices, Meyer lists four critical deficiencies of 
accountability systems that rely on reporting only mean test score performance. First, mean test 
performance is contaminated by factors other than school performance or influence (e.g., student 
characteristics, family, and community factors). Second, reporting mean performance aggregates 
information that is out of date and does not address the cumulative, compounding effect of the previous 
years of instruction. Since reporting mean performance includes all of the students’ previous learning 
experiences, the latest test score actually reflects a summary of a student’s achievement to date. The older 
the student, the more previous instructional efforts account for the student’s current level of performance. 
Third, the mean level of performance is contaminated by student mobility that includes natural breaks in an 
educational cycle (i.e., elementary, middle, and high school) but also movement between schools 
depending on the size of the district and city. Fourth, reporting mean performance on a school basis 
interferes with the ability to localize performance to a grade or classroom level. 

What can be done? 

Given the well-articulated caveats of an accountability system that relies on mean or median test 
scores, one must ask: What can be done? Fortunately, Meyer (1995) and others (Sanders, Saxton, & Horn, 
1997) have proposed using what are now referred to as student gain scores, where achievement scores for 
each student are compared to previous year performance. Thus, each child serves as his or her own control 
(Sanders, Saxton, & Horn, 1997). Meyer posits that growth indicators are good, but an even better model 
is an extension of the growth model referred to as value-added methodology. In essence, value-added 
indices are the most appropriate indicators for measuring school performance by using a statistical 
regression model that includes, to the degree possible, all of the known factors that influence student 
achievement such as student, family and community characteristics (Meyer, 1996). That is, a value-added 
methodology attempts to isolate statistically the unique contributions schools “add” compared to other 
factors related to student achievement. Thus, Meyer and others have argued that failure to account for 
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external sources of student achievement could result in biased or contaminated indicators of school 
performance. 

To summarize, mean level indicators are commonly used and easily understood metrics. They 
provide useful descriptive information, but provide minimal and often misleading evaluative information 
on school performance. Thus, a strong case has been made within the accountability literature to move 
toward growth measures of student and school performance (Darling-Hammon, 1997; Easton, 1994, 
Walberg, 1997). A value-added model goes even further than growth models by including additional 
known variables that are empirically demonstrated to influence student performance outcomes. 

Accounting for these factors is critical since it is well documented that student and family characteristics 
account for far more of the variation in student performance than school-related variables (Meyer, 1996). 

Clearly, it is imperative for school accountability systems to correctly identify programs and/or 
schools that “beat the odds” or “add value.” That is, it is important to empirically distinguish between 
programs or schools that produce results because of their efforts and instruction, not in spite of their efforts 
and instruction. 

The Minneapolis Public Schools Model . 

Any accountability system within education today struggles with balancing the simplicity of 
answering questions such as, “Are programs working?,” and “Are desired outcomes being achieved?” with 
the complexity of accounting for the myriad of factors that influence educational outcomes. For the 
purpose of this paper, we will introduce and discuss a performance continuum model developed by and 
currently used in the Minneapolis Public Schools. In the MPS model we examine and report information 
about: 

• Student level (achievement, growth, and student characteristics) 

• Grade level (within district, school, and classroom) 

• Program level (e.g., special education, ESL, general education, etc.). 

• School-wide interventions (e.g.. Success for A.!!; Discourse; .Accelerated Reader/Math) and, 

• System-level (e.g., Elementary, Middle School, High School, District, Adopted curricula, etc.). 

The Minneapolis Public Schools has worked extensively to build a more equitable and empirically 

sound reporting system to evaluate school performance. The accountability system is encapsulated within 
an entire performance continuum that is described next. The value-added growth model fits into the overall 
framework of placing schools on a continuum from "earning financial rewards” that assist in expanding or 
maintaining excellent performance to "needs prescriptive, corrective action." The latter designation 
includes the option of school reconstitution through the district’s “fresh start” provisions of the teacher and 
principal contracts. The first, “earns financial rewards,” has allowed the MPS to develop Quality 
Performance Awards (financial rewards) that encourage schools to submit applications describing the use 



of best practice coupled with the total points obtained on each of the performance indicators described 
later (see Appendix A). 

The MPS model is a hybrid model that includes 33 indicators for elementary and middle schools 
and 24 indicators for high schools. Both level and growth performance indicators are used. 

Approximately one-third of the performance indicators are considered level indicators and two-thirds are 
classified as growth indicators. The level indicators report point-in-time performance and current level of 
performance on an array of measures. The growth indicators are sub-classified further as: cross cohort, 
gain, and value-added (see Table 1). 

The cross cohort indicators examine change across time, albeit for different groups of students. 
Cross cohort measures include students enrolled at specific sites and compare, across different groups of 
students, the change in indicators such as the number of students passing the Minnesota Basic Skills test, 
change in percent of students performing at certain levels of proficiency on high standard tests (MCA), as 
well as changes in attendance and suspension rates. The gain indices include: percent of continuously 
enrolled students making nationally-normed growth, comparisons among the different racial/ethnic groups, 
accelerated gains at the top and bottom of the achievement distribution, and improvement on oral reading 
measures. Two value-added indicators examine schools that “beat the odds” based on pre-test scores and 
student characteristics (Grade 2 Oral reading and NALT reading and math). 

The ability of MPS to incorporate value-added and gain indicators into a performance evaluation of 
school performance sets the MPS system apart from many other districts. Furthermore, there is an 
excellent literature base to suggest that value-added measures use a more equitable and sensitive analyses 
for examining school effectiveness. Thus value-added analyses are a more appropriate statistic for urban 
educational settings. By incorporating value-added measures, school districts may be better prepared to 
account for important non-school factors that affect student performance and uncover otherwise hidden 
gains. Moreover, districts are more likely to identify schools that are making gains and headed in the right 
direction, a finding that might be masked by a level-based system only. The MPS hybrid system is 
sophisticated enough to isolate particularly effective components so those programs demonstrating gains or 
changes in student trajectory can be clearly identified and replicated in less successful schools. Essentially, 
the MPS hybrid model provides a more sensitive measure to identify improved educational outcomes for 
high-risk students, despite the fact that those students typically exhibit characteristics often negatively 
correlated with achievement. 
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Developing a School Performance Continuum 

How the system evolved . 

Over the past ten years, the Minneapolis Public Schools (MPS) developed a system of data 
collection and interpretation that provides feedback on academic and non-academic school improvement 
indicators. Initial reports in the early 1 990’s included measures of attendance, suspensions, and average 
test scores for a school as a whole. During the 1992-93 school year, a committee of parents, teachers, 
principals, research staff and curriculum experts was convened to develop performance indicators on 
student achievement and indicators on five correlated school improvement areas: school mission and 
vision, leadership, climate, family/community involvement, and effective curriculum/instruction. Baseline 
data for each of the indicators was reported school by school for the first time in 1994. 

In 1995 MPS comprehensive curriculum content standards and grade-level expectations for 
academic achievement were established. The school improvement model that evolved over the next five 
years was developed collaboratively with school district administration, school principal leadership, and 
teacher leadership. Data collection and analysis procedures (e.g., staff and student survey instruments and 
standardized performance assessments) were modified to meet the needs of stakeholder groups. 
Standards-based Assessments . 

Coinciding with these prior events, a team of district curriculum leaders, teachers and research 
specialists was formed to assess the alignment of the district standardized-tests with the newly written 
curriculum content standards. One outcome of these discussions was a recommendation to select a 
different model of standardized assessment. MPS selected the Northwest Evaluation Association (NWEA) 
bank of multiple-choice reading and math items that are pre-calibrated using a Rasch-model. Minneapolis 
curriculum specialists, teachers, and researcher chose items from the item-bank that best aligned with 
district and state curriculum standards. These items were then subject to item bias reviews. Any items that 
were modified from the original format were piloted and re-calibrated. Eleven tests, referred to as 
Northwest Achievement Levels Tests (NALT), were built in reading and 10 tests in mathematics that range 
in difficulty from first grade to 12^ grade level material. 

Using the NALT, each student is assigned to a level of math or reading test (commensurate with 
his or her ability) based on prior test history or a brief "locator" test. Each student then takes a math and 
reading tests where the student is predicted to get 50% to 60% of the items correct. Measurement error is 
reduced significantly over the typical on-grade level assessment where a large percentage of students take 
test items that are too difficult (and thus may resort to random guessing) or too easy (where students may 
rush through simple items and/or be bored with the test). 2 Only students who are severely disabled and 

2 There is practically no ceiling on the NALT item bank. Only a handful of students have achieved a perfect score on 
the highest level test. If a student does get a perfect score on one of the levels tests, they are retested at a higher level. 
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students with severe English language deficiencies are excluded from these tests. Mathematics test scripts 
are also available in seven major non-English languages. With the implementation of levels tests, in each 
grade from 2 through 7 and grade 9, the infrastructure for a growth model of school performance was 
available to the MPS. 

Developing a Quality Performance Framework. 

In 1997 the school improvement model was codified and reinforced with performance incentives. 

A key section of the principal and teacher contracts described the “school performance management 
continuum” (Minneapolis Public Schools, 1997) which included interventions ranging from monetary 
rewards to potential reconstitution of the professional staff at the building. The cornerstone of this school 
performance continuum was a set of indicators designed to be sensitive to growth towards high standards 
of student performance, school climate, professional practice, and stakeholder satisfaction. 

During the 1998-99 school year a Quality Performance Design Team met frequently to determine 
which elements of the larger school information report should be included in the Quality Performance 
Awards. This design team included representation from the principal and teacher bargaining units together 
with District and State accountability experts. After a year of data analysis and discussion, this group 
proposed a set of accountability indices for review by district and community stakeholders. 

In January 2000, a group of district leaders that included the Superintendent of Schools, met with 
representatives of: the Minneapolis Chamber of Commerce, the Minneapolis Foundation (an independent 
non-profit organization), the Minnesota Office of Educational Accountability, and the University of 
Minnesota Department of Education. The primary goal of these meetings was to reach consensus on 
setting evaluative standard criteria for each of the identified school performance indicators. A rubric (i.e., 
1= “well-below expectations” to 5= “well-above expectations”) for each indicator was established. The 
standard setting process set the current expectation at a “3” (or short-term goal to be reached within the 
next 2 years) for all indicators. A “5” equated to well-above expectation and was seen as a long-term or 
stretch goal (i.e., all schools should reach this level within 5 years). 

For example, the district standard for attendance at an individual student level was set at 95% of all 
enrolled days. In other words, a student enrolled for the entire school year may only be absent 8 days if 
they are to meet the standard. The short-term goal for schools was set at 50% of all students attending 95% 
of possible days (currently 44% of the students in Minneapolis are absent 8 days or less per year). The 
long-term goal was set at 70% of students attending at least 95% of possible days. The 1 998-99 school 
year distribution of this indicator is presented in Figure 1 . 
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Figure 1. Distribution of Student Attendance indicator for 1998-99 




Percent attendance @ 95% of possible days 



Standards for each of the 33 indicators used in the elementary and middle school Quality 
Performance Awards and 24 indicators used in the high school Quality Performance Awards can be found 
in Appendix A. National norms and state standards were used to set the standards for academic 
achievement, and discussions among committee members focused on the reasonable length of time to 
expect all schools to reach the standards. Other standards were established by consensus of the committee 
with reference to “common sense” and current baseline distributions. 



Performance Awards 

The Quality Performance Awards (QPA) is an important component of the District’s School 
Performance continuum. The QPA process allows the district to recognize schools for significant growth 
and performance on quality measures and award additional funds. Cash awards, known as Quality 
Performance Awards, were provided to Minneapolis Schools for the first time in 1996. Schools applied for 
the awards based on achievement growth data, attendance, advanced level course participation, and 
engagement in individualized staff development planning. 

The purpose of the QPA process is to maintain a focus on continuous improvement efforts by 
school sites. The additional funds available to the district for rewards are based on a grant obtained by the 
district from the Minnesota Academic Excellence Foundation (MAEF) that in turn is distributed to schools 
demonstrating performance above the district standard on level, growth, and value-added indices. 

The review process required school sites to submit applications to a district committee that 
included district personnel who have specialized expertise in the designated areas that schools describe 
within their best practice narrative (e.g., family involvement). Schools qualified for the awards based on 
their total point value from the 33 indicators listed in Appendix A, in conjunction with a written application 
submitted by the leadership team at the school. On the written application were narratives and data 
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documenting school innovation, behavior intervention plans, staff development plans, parent involvement 
and student activities. The QPA district committee scored the narratives awarding additional points 
following a pre-determined application format (with a 1-5 rubric, see Appendix B). Awards were 
determined at all levels of school configurations: elementary, middle and high schools. A graph depicting 
the distribution of QPA average points for the 71 elementary and middle schools appears in Appendix C. 

In March of 2000, monetary awards from $8,000 to $24,000 were presented to 38 Minneapolis 
Public Schools (approximately 40% of the district schools), depending on the size of the school. The 
indicator framework is a work in progress, but it is also the result of an extensive collaborative process that 
included district principals, administrators, teachers, community members, parents, business partnerships, 
and various community agencies. It is expected that by placing schools along this performance 
continuum — comparing schools to mutually agreed upon standards for each indicator — schools will 
conduct self reviews to: (a) take stock of where they are in comparison to the expected standard (i.e., Do 
they measure up?3), (b) critique the list of indicators, and (c) evaluate the standard-setting process and how 
standards were codified. It is assumed that this process of internal review, as well as external review (by 
the community) will help validate the established criteria for evaluating school performance, which is 
essential for the process to be trusted, open, and embraced by all school sites and staff. 

Indicators for Identifying Effective Schools 

As described previously, the Minneapolis accountability system includes a variety of important 
educational variables. The core indicators include: (1) Student achievement level compared to state and 
district high performance standards; (2) Change in achievement level compared to performance standards 
(i.e., cross-cohort analysis); (3) Student achievement gain compared to expected national norm growth (i.e., 
continuous membership post test - pretest); (4) Student achievement compared to predicted levels of 
performance based on pretest score and student demographics (i.e., , alue-added analysis); (5) Student 
Attendance & Graduation rates; (6) School climate, including safety and respect based on student and staff 
survey responses; (7) Student participation in advanced coursework; and (8) Sufficient course credits by 
year in school for High School students. 



3 Measuring up is the name of the community report card that was released to the community in February 2000 in 
collaboration with the Minneapolis Foundation. 
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Table 1. Elementary and Middle School Achievement Indicators 



Assessment Tool 


Current Achievement 
Level 


Cross-Cohort Change 
in Achievement Level 


Achievement Gain 
& Value-added 


. ' . vv*' l . ■ 


Indicators 


Indicators 


v Indicators. . ■ 


Northwest 
Achievement 
Levels Tests in 
Reading and 
Math in Grades 
2-7 and 9 


Percentage of students 
on course to pass the 
Minnesota Basic 
Standards Tests 




• Percent of students 

making national norm growth for 
lyr and 3yrs 

• Equity of growth rates across 
racial/ethnic groups 

• 20/20 Analysis of growth 
at the top and bottom of the 
achievement distribution 

• NALT Value-added: Spring 
scores compared to prediction 
based on previous spring test 
scores and demographic factors 


Oral Reading 
words read 
correctly per 
minute in grade 
level text 






• Grade 2 oral reading rate 
versus prediction based on Grade 
1 oral reading scores and student 
demographics 


Minnesota Basic 
Standards Tests 
(reading & math) 
8th . 12th grade 


Percent of students 
passing the tests (given in 
8th grade & High School 
until students pass) 


Change in the percent 
of students passing the 
test from year to year 




Minnesota 
Comprehensive 
Assessments (in 
Reading, Math & 
Writing) Grades 
3 and 5 


Percentage of student in 
four competency levels 
converted to a point 
system. Points compared 
to state and district high 
standards. 


Change in points for 
movement of students 
from one competency 
level to another 
(includes new students) 




Attendance 


•Percent of students 
attending 95% or more of 
possible days 
•Percent of students 
attending summer school 


Yearly change in 
percent of students 
attending 95% or more 
of possible days 




Suspensions 


Percent of students 
suspended for aggressive 
behavior 


Change in percent of 
students suspended for 
aggressive behavior 




Climate 


Student & Staff Survey 
responses for safety and 
respect 






Gifted and 
Talented 


Number of quality 
components found in 
audit of the program 
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Current Level of Performance Indicators and Poverty Levels 



Level indicators from point-in-time assessments of academic achievement are confounded with 
entry level achievement of the student. It has been well established that achievement test scores without 
consideration of the entry-level scores are highly correlated with poverty levels. In Minneapolis, 
correlations between point-in-time Northwest Achievement Levels Test (NALT) scores and the percentage 
student receiving free or reduced price lunch have been consistently very high. Figure 2 presents the 
correlation between the 1999 NALT test of mathematics and the percentage of students receiving free or 
reduced price lunch in Minneapolis elementary and middle schools (n=71). It shows that 79% of the 
variance in school average test scores is accounted for by the percentage of students on free or reduced 
price lunch. Also noted in the graph is School P that has 93% students on free or reduced price lunch and 
an average NALT score at about the 40^ national percentile (nce=44.4). Using a level system approach 
this school would be in the bottom l/3 f d of the distribution. Thus, school “P” is actually performing above 
the trend line, a fact that would be masked without closer inspection and appreciation of the effects of 
poverty on entry level skills. 

Figure 2. Distribution of school average 1999 Northwest Achievement Level Test 
mathematics scores by free or reduced price lunch percentages. 




Percent free or reduced price lunch 



Growth Measures 

Cross-cohort change . 

Cross-cohort indicators are the first of three growth indicators used within the MPS performance 
continuum. This type of indicator is typical for assessments that are given periodically at certain grade 
levels. For example, the Minnesota Basic Standards Tests are given for the first time in 8*h grade. Scores 
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are reported each year for the district and schools, comparing the current year 8th-grade cohort to the 
previous year 8th-grade cohort. Arguably, while cross-cohort measures may be a better school 
performance indicator than the average test score at one point in time, they too can be prone to 
misinterpretation. For example, when entry-level skills of students vary from year to year or when a 
specific site experiences a dramatic shift in its student population, cross-cohort analyses can be highly 
misleading. Additionally, large changes in the percentage of students passing a test from one year to 
another may occur when special programs are added to a school, when school attendance boundaries 
change, or due to random fluctuations in the preparedness of students enrolling in the school. Obviously, 
cross-cohort analyses require more sophisticated interpretation. Despite these caveats, the Minnesota 
Comprehensive Assessments are reported only at grades 3, 5 and 10, and the Minnesota Basic Standards 
tests at grade 8, thus a cross-cohort analysis is the only form of growth data available at this time^. 

Gain Indicators. 

Student pre-post gain measures and value-added indicators are the backbone of the district 
accountability model. NALT scaled score gains are followed longitudinally for each student enrolled in the 
school district, and form the basis forjudging whether each student is making adequate yearly progress 
toward district and state high standards of performance. This accountability indicator is one of the most 
frequently cited indicators of school performance within School Improvement Plans. Simply stated, 
schools can report on the percent of students making at least one-year’s gain in one-year's time. By 
comparing each student's yearly scaled score gain versus the Northwest Evaluation Association growth 
norms, school effectiveness can be measured "one student at a time.” In fact, this indicator has been found 
to be easily understood by most stakeholders compared to average scaled score growth or NCE gain. 

The scaled scores obtained from the district NALT administration are used to determine individual 
student growth curves and to report the percentage of students making gains equal to or greater than the 
grade level norms. The NALT developers provides information on rate of growth in reading and math 
compared to the user group norms for over 500,000 continuously enrolled students. Obtained scaled score 
growth is compared to these national user norms to aggregate the percent of students in the school who 
make at least one year’s growth in one year’s time. 

Additionally, scaled scores from the NALT have been equated with Minnesota Comprehensive 
Assessments (MCA) in grades 3 and 5 and the state graduation test in grade 8. The obtained correlation 
between state test scaled scores and NALT scaled scores at the same grade (i.e., concurrent validity 
coefficients) and at previous grade (i.e., predictive validity) are very high as shown in Table 2. All 
correlations are for the same subject (i.e., NALT reading test with MCA reading test, etc.). 



4 However, the vendor for state testing in Minnesota is doing vertical equating so that gain scores can be reported on a 
continuous scale in the future 
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Table 2. Northwest Achievement Levels Tests (NALT) Concurrent and Predictive Validity 

Coefficients for all students assessed with Minnesota Comprehensive Assessments MCA) in 
Grades 3 & 5 and the Minnesota Basic Standards Tests (MBST) in Grade 8. 



Grade/Measure 


Grade 3 
Reading 
MCA 


Grade 3 

Math 

MCA 


Grade 5 
Reading 
MCA 


Grade 5 

Math 

MCA 


Grade 8 
Reading 
MBST 


Grade 8 

Math 

MBST 


Grade 2 NALT a 


(n/a) 


.80 

(n= 3219) 










Grade 3 NALT b 


.87 

(n=3785) 


.87 

(n=3640) 










Grade 4 NALTa 






.85 

(n= 3153) 


.86 

(n=3097) 






Grade 5 NALT b 






.88 

(n=3533) 


.89 

(n=3484) 






Grade 7 NALT C 










.82 

(n= 2608) j 


.88 

(n= 2631) 



a NALT April, 1998 correlation with MCA March, 1999 for same subject area (predictive validity) 
b NALT April, 1999 correlation with MCA March, 1999 for same subject area (concurrent validity) 



C NALT April, 1998 correlation with MBST February, 1999 for same subject area (predictive validity) 

Moreover, student performance on district measures can also be converted to normal curve 
equivalents for purposes of reporting overall school mean gains. Normal curve equivalent (NCE) scores 
are non-linear transformations from the percentile distribution that provide an equal interval scale for cross 
grade aggregation and comparison to national norm expectation. In the standard normal distribution the 
mean NCE = 50 and the standard deviation is 21 .06. Mean NCE gains have been used in Minneapolis 
since the early 1990s for comparison among schools, racial/ethnic groups, and family income levels (e.g., 
free or reduced price lunch). While current level of performance by school is highly correlated with free or 
reduced price lunch percentages (Figure 2), one-year gains on the Northwest Achievement Level tests are 
relatively uncorrelated with measures of SES at the school level as indicated in Figure 3. 

For example, additional analyses indicate that the free and reduced lunch percentage indicator 
accounts for less than 1% of the school by school variance in average mathematics gain. Please note that 
School “P” referenced earlier (see figure 2) had the highest average gain in math scores from 1998 to 1999. 
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Figure 3. Distribution of school average mathematics gain scores on the 1999 Northwest 
Achievement Level Test by free or reduced price lunch percentages. 
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Program success, as noted here for School “P” (Figure 3) becomes unmasked by accounting for 
student characteristics associated with performance and then examining gain that “beat the odds.” Data for 
the past three years of testing are reported as one of the performance indicators as well. Thus, schools that 
have made significant progress over three years, but due to normal fluctuations in scores are credited for 
overall school performance. 

Individual and group growth curves (Figure 4) depict student progress towards the Minnesota 
Basic Standards Test, a test necessary to graduate (that are roughly equivalent to the national norm in 8^ 
grade) and high standards tests (MCA, which are approximately equivalent to the 80^ national percentile 
rank at each point of administration). 
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Figure 4. One student’s achievement on Northwest Achievement Level Reading Test versus 
the growth curves for the Minnesota Basic Standard (graduation test) and 
Comprehensive Assessments (high standard) estimates 




Finally, gain scores on the NALT (grades 2-7) are disaggregated by racial/ethnic group and growth 
rates analyzed to determine whether the growth rate for each group is sufficient to reach the basic standard 
level within five years. This approach addresses disparities of achievement by emphasizes "gaps to the 
standards " rather than the traditional approach of gaps between different racial/ethnic groups. In the 
traditional approach, it is possible to narrow the racial/ethnic gap by reducing growth for one group of 
students. Instead maximum points are awarded to schools who show accelerated growth toward high 
academic standards for all groups of students (see figure 5). 

In this example, Emerson earned all five points on the indicator for achievement equity since all 
racial/ethnic groups enrolled recorded average reading gains that are sufficient to reach the Minnesota 
Basic Standard level in 5 years or less^. 




5 Note that expected growth in nee units = zero; A straight line indicates that students on average have maintained 
their relative status versus the national norm, which in this case is approximately equivalent to the state 8 th grade 
standard. 
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Figure 5. Northwest Achievement Level Gain 1998-99 and projected linear progress for Emerson 
Spanish Immersion Elementary School. 




20/20 Analysis 

The Minneapolis Public School’s Research, Evaluation, and Assessment department also employs 
a 20/20 analysis, coined by Reynolds and Heistad (1993). Since schools are given the laudable charge to 
ensure all students learn, it is critically important, if not imperative, that schools are provided with 
measures of student performance that address the full continuum of student skill and ability level. Thus, 
schools need information that ensures all students are “visible” within any devised accountability system 
that reports on student performance. Therefore, a 20/20 analysis ensures that students at the margins, as 
well as for students who are performing at the central tendencies, are included in data analyses and 
available for school-level planning. 

A 20/20 analysis is the examination of continuously enrolled students^ across time that isolates 
students performing at the top and bottom (1/5™) margins of a distribution, as well as students at the 
“average” or median level. The rationale for focusing on students at these identified margins is that these 
are the students who most clearly require adaptation in instruction that accounts for their exceptional status 
high-above or well-below grade level norms (Reynolds & Heistad, 1997; Reynolds, Zetlin, & Wang, 

1993). 

By having information at hand that allows schools to examine the trends for each of these student 
levels of performance, it can be determined if a program, school, or district has become “specialized” in the 




6 For school-level accountability it is imperative to examine the performance of students who had the opportunity to 
benefit from or the misfortune of being enrolled at a specified site. 
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sense of accelerating learning within one level of student performance. For example, it is conceivable that 
a school program, building, or district could become so focused in their deployment of resources and 
delivery of instruction that they become overly specialized in meeting the instructional needs of low- 
achieving students. This in effect would create an unintended negative consequence of failing to meet the 
instructional needs of high-achieving students. Thus, a 20/20 analysis maintains the focus and reporting of 
student performance across the full continuum of student ability and skill level and avoids creating a 
reporting system that masks discrepancies in student performance among different skill levels. 

Furthermore, the 20/20 analysis has been used as an excellent tool to review additional 
programming opportunities for students at the extremes. For example, Reynolds, Zetlin, and Wang (1993) 
conducted an examination of students who were performing at the extremes and the types of programming 
being afforded to students (e.g., gifted and talented, Assurance of Mastery, Title I, ESL, or special 
education, etc.). In Minneapolis Schools, 85% of the students in the bottom 1/5^ of the score distribution 
were receiving special programming, and 75% were enrolled in special education (Heistad & Reynolds 
1995). Moreover, this type of exploration revealed additional information on characteristics highly related 
to student performance, such asstudent attendance. In fact, students in the bottom 1/5^ of the distribution 
had an absenteeism rate two times higher than the school average. In sum, the 20/20 analysis provides a 
unique metric for evaluating the effectiveness of programming established to address student 
exceptionality. 

In the 20/20 example that follows, information is reported for one school Emerson Spanish 
Immersion Learning Center (SILC). Data are reported by identifying specific cut-points in the 
achievement distribution that separates out the top 1/5^ and bottom 1/5^. The greatest number of Quality 
Performance points (5 points) are awarded to schools where the top and bottom cut points have shifted 
upward compared to the national norm. 

For example, in Figure 6 the distribution of NALT scores fc. continuously enrolled students 
(N=l 86) at Emerson SILC is depicted by plots for the bottom 1/5^ cut score, the median, and the top 1/5^ 
cut score in 1998 and 1999. In the case of Emerson SILC, the entire distribution of scores shifted from 
1998 to 1999. The cut point for the bottom l/S^ 1 increased from a normal curve equivalent (NCE) of 30 to 
35 and the cut point for the top 1/5^ increased from a NCE of 73 to 78. This amount of growth in one 
year is about 1/4^ of a standard deviation on the normal curve, which is a substantial gain (note: the NCE 
distribution has a standard deviation of 21). 
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Figure 6. Emerson Spanish Immersion 20/20 Analysis NALT Reading Scores Continuously 
Enrolled Students (N=186) From spring 1998 to spring 1999 
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Overall, the 20/20 Analysis has been established as an efficient and useful metric for analyzing the 
progress of students and of evaluating the programs provided for them (Reynolds & Heistad, 1997; 
Reynolds, Zetlin, & Wang, 1993). It is our assumption that this analysis, in conjunction with the additional 
analyses described in this paper, can enhance instructional approaches for all students, as well as maintain 
a wide perspective in analyzing the effectiveness of these approaches. 

Value-Added Indicators. 

In Minneapolis Public Schools, “value-added” multiple regression analyses are conducted to 
determine school effectiveness. Like gain measures, value-added indicators report average growth from 
pretest to post-test for students enrolled in the school for at least one year7 Unlike simple gain, the value- 
added corrects for the problem of correlation between initial test score and gain by using the pretest as a 
predictor of posttest performance. The value-added regression procedure used in Minneapolis also 
includes the following predictors^: 

1) Free or reduced price lunch 

2) English Language Learner (ELL, formerly LEP) status 

3) Special Education status 

4) Gender 

5) African American and American Indian status 

6) Lives with both parents status 

7) Lives in high poverty zip code status 

One method of calculation of value-added school effects computes the average residual from the simple 
regression across schools. This method is coined the “beat the odds” approach. All coefficients are 
centered around zero and represent the contribution of the school to student achievement greater than 
predicted. Another method of calculation substitutes actual district percentages of free or reduced lunch, 
ELL, Special Education, etc. into the obtained regression equation to estimate school by school 
performance for the “typical classroom” in the district. 

Value-added indicators are the most equitable indices in comparing schools w^h differing 
populations of “at-risk” students. However, they are also the most difficult indicators to explain to the 
general public. As the public becomes accustomed to the presentation and interpretation of value-added 
effects it may be more feasible to add more value-added indices. 

The Minneapolis Public Schoofs Improvement Framework . 

Increasing accountability among schools within the Minneapolis Public School system (MPS) is 
the primary goal of the performance continuum framework. Inherent in any system as detailed and 



7 Minneapolis is working with Rob Meyer at the University of Chicago to incorporate students with less than one year 
attendance into the value-added model. Pretests given to students at the time of admission to the school will allow a 
computation of partial year growth. These tests are currently in the pilot stage. 

8 All of these factors are “dummy coded” with “0” or “1” in a single stage multiple regression with all factors entered 
simultaneously. For example “1” = current special education IEP; “0” non-special education. 
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complex as described here is the need to provide schools with a continuous cycle of information relative to 
the performance indicators, and then a method for reviewing, evaluating and setting new goals. The MPS 
district is building a multifaceted accountability system that encompasses individual, school, and system 
accountability to improve student learning. At the individual level, student accountability is being forged 
in the form of curriculum content and performance standards that are aligned with standards-based 
assessments. The district uses a standardized item bank with multiple test levels for each grade-level, 
which maximizes the opportunity to include most if not all students in the accountability system. The 
purpose is to measure student growth toward high academic standards as well as current status relative to 
those standards. 

Next, we describe the Minneapolis Public Schools accountability cycle in relationship to the 
academic calendar year. 

Accountability Components. 

The district accountability model consists of six main components: 

• School Performance Continuum 

• School Improvement Reports (SIR) 

• School Improvement Plans (SIP) 

• School Improvement Plan Feedback (SIF) 

• Quality Improvement Process (QIP) 

• Quality Performance Award (QPA) 

These accountability tools are different facets of a single continuous improvement process for each 
school site^. The framework is aligned with the Strategic Direction of the Minneapolis Public Schools, the 
District Improvement Agenda, curriculum and performance standards for learning, and school quality 
standards. This integrated system should help to enhance and maintain consistent feedback to schools 
about expectations for improvement related to clearly articulated District standards. In addition, the 
performance continuum should help to guide the work of those providing feedback to schools, such as 
school improvement feedback (SIF), school visits by a quality cadre (QIP), school improvement reports 
(SIR), and monetary performance awards (QPA). 

The Accountability cycle . 

The MPS model of provides continuous feedback on student performance, attendance, building 
climare, etc. and supports teachers and the site leadership teams to openly discuss priorities for allocating 
resources and adapting instruction to enhance student achievement. This process is called the School 
Improvement Planning Cycle, depicted in Appendix D. Beginning in August, critical data regarding 
student performance is included in an annual School Information Report (SIR) provided to principals, 
teachers, parents and other leadership team. These reports include data such as: 




21 

23 



• student performance on state high-stakes graduation assessments, the Minnesota Basic 
Standards Tests (MBST). 

• student performance on high standards measures, referred to as the Minnesota Comprehensive 
Assessments (MCA). 

• scaled score information on the Northwest Achievement Levels Test and school- and grade- 
level information on the percentage of students “on-course” to pass the MBST in the content 
areas of math and reading. 

• staff and students responses to a district administered survey. 

• staff and student characteristics. 

The school Information Report also provides math and reading gain scores by grade level, referenced to 
national growth norms for students who are continuously enrolled across two testing periods (e.g., spring to 
spring from one year to the next). 

At the school-level, each site is encouraged to recruit key stakeholders from inside and outside of 
the school building (e.g., staff, parents, administrators, site council members, business community, etc.) to 
critically review the reported progress of the school in relationship to the identified indicators. These 
critical reviews are a way to report back to interested stakeholders about school performance and to 
examine the strengths and weaknesses of the current programming so that areas of need may be prioritized. 
Once areas of need are prioritized, school teams are assigned the task of developing a SIP. SIPs are used 
to carefully craft the short-term and long-term goals and objectives a school will put in place to affect 
change. Additionally, SIPs address how resources will be allocated to support the identified initiatives at 
each site. Finally, schools must identify a system of continuous measurement to ensure that mid-course 
corrections can be made —if sufficient progress is not made-- as well as to allow definitive statements about 
overall school outcomes. The SIP, in essence, becomes the school’ s “road map” for charting an academic 
path and documenting how information will be gathered and used to evaluate overall performance. 

After each school site completes their SIP, they submit the document to the district office where a 
team of objective readers is convened to review the documents for clarity, cohesiveness of school-based 
initiatives, and appropriateness of the methods selected related to the target goals and objectives. Written 
feedback is provided by the external reviewers and is provided back to the building site teams in a face-to- 
face meeting referred to as School Improvement Plan Feedback (SIF). The SIF meetings occur throughout 
the academic year beginning in late fall. In addition, schools are encouraged to provide progress reports 
and up dates on student progress at these meetings. 




See appendix for a flowchart of the linkaj 
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In the spring of each academic year, district assessments in reading (grades 1-7, 9) and math 
(grades 2-7, & 9) are administered along with district-wide staff and student surveys. Information is 
reported back in late spring to allow individual reports to be generated for students, families, teachers and 
staff. These reports allow schools to review the progress made to date in relationship to the SIP developed 
earlier in the academic year. 

The MPS Research, Evaluation, and Assessment department then begins the task of aligning and 
reporting on all 33 or 24 indicators for the August SIR released to the schools and community. The REA 
staff work to ensure information is reported using the identified level and growth indicators and to study 
the relationship to external measures of progress (State high stakes measures and measures of high 
standards, and national exams). MPS is working on maintaining the SIP cycle and helping schools to 
extend their focus toward more long-term planning, and to provide assistance to schools in trouble so that 
they may make more immediate corrective actions. The school improvement cycle is a framework that 
empowers leadership at individual schools to maintain and expand innovative programs and to make 
corrective action when necessary. The performance continuum described within this paper describes the 
ability to reward schools with stellar performance. In extreme cases, when schools fail to demonstrate 
expected levels of performance or gains, the district will step in and become more prescriptive, providing 
more intensive support services to affect school improvement. 

Much of the discussion up to this point has focused on school and district indicators as well as 
processes of accountability. However, Minneapolis has begun to expand their use of value-added analyses 
to focus attention on teachers “who beat the odds.” In this way, the district can begin to cast an eye towards 
replicating successful practices rather than using these indicators to identify “low performing” teachers. A 
study of teacher value-added effects in second grade reading provided feedback on the instructional 
methods associated with exceptional instruction (c.f., Heistad, 1999). The goal is to help identify 
exceptional practices and provide opportunities to replicate these effective practices across the district. 

The primary goal of developing and deploying a performance continuum of school performance 
has been to ensure continuous improvement among all Minneapolis Public Schools as well as to ensure 
continuous improvement in student learning. The performance continuum described in this paper is 
obviously a work in progress. The critical question for any district to answer is this: Are the identified se t 
of indicators providing a substantially better measure of school performance than other affordable 
indicators ? And do the indicators in concert make a net, positive contribution to the school improvemen t 
process, relative to other possible indicators and school accountability systems available ? (Meyer, 1 999, 
p.8-9). As a district committed to high standards and good academic progress of all its students, we affirm 
that indicators of school performance which include growth and value-added indices are essential metrics 
to include in any fair, equitable, and non-biased system of evaluation. 
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Findings and Further Discussion 

At the school level, no one would disagree that student characteristics such as poverty and prior 
achievement levels are related to mean school performance. Furthermore, previous analyses indicate at the 
student level, that prior test scores, racial/ethnic background, and special educational needs (LEP and 
Special Ed.) affect student learning. However, some models of school performance evaluation do not 
explicitly account for student demographic differences in their value-added model (e.g., see a review of the 
Tennessee Value-Added Assessment System, Bock, 1996). In MPS previous experience and extensive 
statistical modeling (Du & Heistad, 1999; Meyers 1999, 2000) has indicated that a multiple regression 
approach that accounts for poverty, race, special education, and English language status does provide more 
precise estimates of school performance, compared to a model of post-test performance controlling for 
pretest. Additional studies and comparisons of the different statistical models are currently underway 
(Meyer & Heistad, in progress). 

This paper describes Minneapolis Public Schools accountability system and presents the indicators 
used in the system. Moreover, this paper provides a practical application of how value-added indicators can 
and should be included in any accountability model designed to evaluate the effectiveness of schools. The 
Minneapolis Public Schools model proposed here reflects school performance relative to pre-agreed upon, 
pre-selected indicators and clearly articulated standards of performance. These standards include State of 
Minnesota Comprehensive Assessments and graduation tests reported each year. The cross-cohort 
analyses compare changes in the percent of students performing at different proficiency levels or passing 
graduation tests. Yet, we have argued in this paper that current level of performance and "cross-cohort" 
analyses of achievement are relatively weak indicators of overall school performance and should be 
supplemented with indicators that follow students progress across time. The variety and number of gain 
indicators and value-added indicators of academic performance are what we believe distinguish the 
Minneapolis Public Schools accountability system from those that largely focus on current level of 
performance. 

Current level of performance and cross-cohort indicators are typically unfair to schools with high 
mobility and school enrollments with larger percentages of at-risk students. As large number of language 
minority students and students with other special needs enter the school district, certain schools are much 
more likely to receive disproportionate numbers of students with low entry-level skills. In comparison 
other schools might have different enrollment patterns and be more likely to receive students with high 
entry-level skills. Current level accountability systems would not be equitable in providing unbiased 
evaluations of school performance. 

The present set of indicators used in MPS is based on extensive collaboration among community 
stakeholders, district leaders and research staff. The MPS hybrid model that was approved for this year’s 
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performance awards includes more growth indicators than some stakeholders wanted and not enough for 
others. Similarly the balance between academic and non-academic indicators was the result of compromise 
among the stakeholders. Each indicator that was chosen also needed to pass the “gas pump” criteria. That 
is, could the indicator be explained to an ordinary tax payer in five minutes, while filling up the gas tank. 
Some indicators, like the 20/20 analysis and value-added approaches need to be summarized with 
communication "sound-bites.” 

For example, in the 20/20 analysis “we are making sure that all students, not just average 
performing students, are making progress towards the high standards.” When describing the value-added 
analysis, we talk about “acknowledging schools that beat-the-odds: schools who are doing an exceptional 
job with students in poverty, students of color and special needs students.” Clearly some high stakes 
accountability systems have failed to pass the "gas pump" test in the eyes of their critics (Fisher, 1996). 

Ultimately, the value of this accountability system will be measured in terms of its consequences. 
That is, we must evaluate the extent to which interventions can be matched to the information provided to 
schools via the indicator system to improve student learning. To this end, a multiple indicator system may 
again be helpful. Not only do multiple achievement indicators provide increased reliability of 
measurement, each of the indices provides some form of diagnostic feedback to the school. 

Furthermore, there is reason to believe that effective accountability systems emphasize reward 
rather than punishment. Principles of behavior intervention suggest that punishments are likely to result in 
emotional behaviors (e.g., fear and resentment) and are unlikely to advance learning of new behaviors 
(Skinner, 1953). In the extreme situation, school leadership and faculty may react very aggressively 
towards the accountability system where the school actually expects that they should be receiving a reward 
for their efforts and instead receive threats of or actual punishment. Consider the case of School “P” 
depicted in Figures 2 and 3. At the end of the 1998-99 school year their average level of mathematics 
achievement placed them well below the basic standard level — they are in the bottom third of all 
elementary schools in the district. However, this school demonstrated one of the highest mathematics 
learning rates over a one-year period. Under some accountability systems school “P” might have been 
reprimanded, in Minneapolis this school had multiple indicators of value-added and received a Quality 
Performance Award. 

Finally, the "hard data" from summative assessment should not be the only thing that matters. 

When schools receive detailed school information reports from the research department in the fall of each 
year, they revise goals and strategies for the coming year. However, they bring a wealth of school level 
data to the table as well. If the accountability system indicates a weakness in one particular area of 
mathematics, the school may turn to building level performance assessments and permanent products of 
student work to localize the deficiencies in certain grade levels or classrooms. If reading scores are down. 
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formative assessments of individual student growth rates on classroom level assessments may provide a 
framework for evaluating new strategies. 

When the MPS Area Superintendent engages in a "performance conversation" with the leadership 
team at a particular site, school level data on such things as the degree of program implementation and 
formative evaluation data must play at least as prominent a role as the summative data on indicators of 
progress. If reconstitution 10 0 f the school is to be considered, a detailed audit of variables underlying the 
accountability indices should also be undertaken. This is a time when professional judgment needs to 
supersede the statistical index. Hopefully, this paper will be useful to practitioners, administrators or 
researchers interested in identifying and combining school performance indicators to make high stakes 
accountability decisions for school districts. 



10 Further studies need to examine the efficacy of reconstitution a popular, but unproven intervention for increasing 
student achievement. 
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Cover Sheet 



Quality Performance Award Criteria 

(Possible points shown in right column, awarded points in lowermost right.) 



Leadership Sign-Off 


Pts 


Represents all stakeholder groups (parents, community 
rep, students, if age appropriate), teachers, principal) 
on site team; staff comprises 50% or less of team 


5 


Represents all stakeholder groups (parents, community 
rep, students, if age appropriate), teachers, principal) 


3 


Leadership learn Chair, principal, teacher and parent 
represented, some stakeholder groups not represented 


1 


Points Awarded 





Participation of Others in Application 


Pts 


PDF Coordinator, Stall Development Coordinator, 
community representatives and others in collaborative 
roles 


5 


FDP Coordinator and Stall Development Coordinator 


J 


Others in collaborative roles 


1 


Points Awarded 





Professional Development Plan Alignment 


Pts 


Alignment, active teams and an array of activities 




Alignment, with active teams (meet more than 6 
times/year) 


3 


Alignment with School Improvement Flan 


1 


Points Awarded 





Written Application: 



A. 1 & 2 School Improvement Goals 


Pts 


Surpassed “stretch” goal aligned with D1A 


15 


Surpassed goal 


9 


Achieved goal 


~T~ 


Points Awarded 





A. 3 Behavior Plan 


Pts 


Behavior Flan completed and submitted, with data and 
strategies clearly linked 


5 


Behavior plan completed and submitted 


~r“ 


Points Awarded 





A. 4 &5 Staff Development Plans 


Pts 


“"Plan clearly aligned with SIP with goals that include 
clear, well-defined indicators 


5 


Coals clearly related to SIP 




Flan developed, inconsistent alignment with S1F 


i 


Points Awarded 





A. 6 Communitv/Parent Involvement 


Pts 


Significant increases in community and parent 
involvement demonstrated using multiple measures 
promising strategies implemented, frequent monitoring 


5 


Increases in community and parent involvement shown 
on more than one measure 


~~ r~ 


Planning and strategies implemented to increase 
community and parent involvement 


i 


Points Awarded 





A. 6 School/ Parent Involvement 


Pts 


Significant progress demonstrated on multiple indicators 
of school level Family Involvement Standards 


5 


Progress demonstrated on multiple indicators ot school 
level Family Involvement Standards 


3 


bvidence that some indicators ot school level Family 
Involvement Standards monitored 


1 


Points Awarded 





A. 6 Classroom Parent Involvement 


Pts 


Significant progress demonstrated by a number ot 
classrooms on indicators of classroom level Family 
Involvement Standards 


5 


Progress demonstrated on multiple indicators ot 
classroom level Family Involvement standards 


3 


bvidence ot a some indicators ot classroom level Family 
Involvement Standards monitored 


~^r~ 


Points Awarded 





A. 7 Student Involvement in Activities 


Pts 


Significant progress demonstrated on indicators of 
student participation in co-curricular (i.e., after school) 
activities 


5 


Progress demonstrated on indicators ot student 
participation in co-curricular (i.e., after school) activities 


3 


Evidence that levels of participation in student activities 
is monitored 


1 


Points Awarded 





A. 8 Innovation 


Pts 


Promising strategy developed and implemented; 
networking potential evident; funded from site allocation 


i 5~~ 


Promising strategy implemented, networking potential 
evident, funded from grant or district source 


~^r~ 


Promising strategy implemented 


l 


Points Awarded 





B. 1 Plans for Use of QPA Funds 


Pts 


includes specific plans to collaborate with other sites 
and networking shows promise to deliver on outcomes 
aligned with district’s priorities, e.g., middle school 
redesign, transitions 




Plans to share promising practices with staff and 
community, e.g. open houses, fairs, electronic 
presentations on web, or products will be produced for 
presentations 


— 3 — 


Plan is integrated with SIP and D1A 


1 


Points Awarded 





B. 2, C. 1 Previous Award Use 81 Networking 


Pts 


Networking impacted another site's improvement 


~ 5“ 


Award used to support continuous improvement goals at 
recipient site, e.g., student work activities provided, 

PDP supported, leadership developed 


3 


SIP goals supported 




Points Awarded 
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Average QPA Rating for 71 Minneapolis 
Elementary and Middle Schools 

30 
25 
20 

Number of 
Schools 

10 
5 
0 

1.5 to 1.9 2.0 to 2.4 2.5 to 2.9 3.0 to 3.4 3.5 to 3.9 4.0 + 
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members of the education community who could not attend your session or this year's conference. 

Abstracts of papers accepted by ERIC appear in Resources in Education (RIE) and are announced to over 
5,000 organizations. The inclusion of your work makes it readily available to other researchers, provides a 
permanent archive, and enhances the quality of RIE. Abstracts of your contribution will be accessible 
through the printed, electronic, and internet versions of RIE. The paper will be available full-text, on 
demand through the ERIC Document Reproduction Service and through the microfiche collections 
housed at libraries around the world. 

We are gathering all the papers from the AERA Conference. We will route your paper to the 
appropriate clearinghouse and you will be notified if your paper meets ERIC's criteria. Documents 
are reviewed for contribution to education, timeliness, relevance, methodology, effectiveness of 
presentation, and reproduction quality. You can track our processing of your paper at 
http://ericae.net. 

To disseminate your work through ERIC, you need to sign the reproduction release form on the 
back of this letter and include it with two copies of your paper. You can drop of the copies of 
your paper and reproduction release form at the ERIC booth (223) or mail to our attention at the 
address below. If you have not submitted your 1999 Conference paper please send today or 
drop it off at the booth with a Reproduction Release Form. Please feel free to copy the form 
for future or additional submissions. ... ...... . • .« — , . . - ... 

Mail to: AERA 2000/ERIC Acquisitions 

The University of Maryland 
1129 Shriver Lab 
College Park, MD 20742 



Sincerely, 




Lawrence M. Rudner, Ph.D. 
Director, ERIC/AE 



Tel: (800) 464-3742 
(301) 405-7449 
FAX: (301) 405-8134 
ericae@ericae.net 
http://ericae.net 



ERIC/AE is a project of the Department of Measurement, Statistics and Evaluation 
at the College of Education, University of Maryland. 




